Examples of encoding formats to encode images with multiple viewpoints, such as 3D (Dimension) images and the like include the MVC (Multiview video Coding) format which is an extension of the AVC (Advanced Video Coding) (H.264/AVC) format, and so forth.
With the MVC format, images to be encoded are color images having values corresponding to light from a subject, as pixel values, with each color image of the multiple viewpoints being encoded, referencing color images of other viewpoints as appropriate as well as to the color images of those viewpoints.
That is to say, with the MVC format, of the color images of the multiple viewpoints, the color image of one viewpoint is taken as a base view (Base View) image, and the color images of the other viewpoints are taken as dependent view (Dependent View) images.
The base view image (color image) is then encoded referencing only that base view color image itself, while the dependent view images (color images) are encoded referencing images of other dependent views as appropriate, besides the image of that dependent view.
Also, a method for separately encoding a color image and a disparity information image has been proposed as a method for encoding the color images of multiple viewpoints, and a disparity information image having disparity information (depth information) regarding disparity for each pixel of the color image of each viewpoint as a pixel value (e.g., see NPL 1).